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© Personal computer with generalized data streaming apparatus for multimedia devices. 
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© A multimedia data processing system includes a 
memory for storing multimedia application programs 
and a multitasking operating system. Extensions to 
the operating system control data streaming from 
source devices to target devices to provide real- 
time, continuous streaming. The extensions provide 
central buffer management with a user buffer option, 
bi-level priority support for data stream handlers, 
support for interleaved streams, and data stream 
event detection and notification. 
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Field of the Invention 

This invention relates to the field of data pro- 
cessing, and, more particularly, to a personal com- 
puter based multimedia system in which data 
streaming occurs in a continuous, real-time fashion 
under the control of a multitasking operating sys- 
tem. 

R elated Application 

Matter forming the subject of the present pat-, 
ent application also forms the subject of our Eu- 
ropean Patent Application claiming priority from our. 
application filed in the United States of America on 
31 December 1992 under Application No. 815,652. 

Backgroun d of the Invention 

A multimedia system is designed to present 
various multimedia materials in various combina- 
tions of text, graphics, video, image, animation, 
sound, etc. Such a system is a combination of 
hardware and software. The hardware includes a 
personal computer to which various multimedia de- 
vices can be attached. The hardware runs under 
the control of an operating system and multimedia 
application programs. 

Multimedia applications impose heavy de- 
mands on the operating system to move large 
amounts of data from device to device, from sys- 
tem memory to a device, or vice-versa, in a con- 
tinuous, real-time manner. Multimedia systems 
must support a flexible yet consistent means for 
transporting these large data objects, and control 
this activity accurately in real time. Adding new 
multimedia devices and data types should require 
minimal, if any, new system extension code. The 
total real memory requirements at run time must 
be minimized, so that system performance is not 
degraded. Also, support for complex data types 
and devices that manipulate interleaved data ob- 
jects, must be provided. Finally, it must be possible 
to implement each multimedia data transport con- 
trol means at the most appropriate system privilege 
level. 

Operating system extensions that support mul- 
timedia data types and devices must provide the 
ability to control the many different multimedia I/O 
devices and to transport, or stream, large mul- 
timedia data objects within real-time constraints. 

Multimedia applications control the input and 
output of large amounts of data, including the con- 
tinual display of bitmaps, output of digitized audio 
waveforms or MIDI audio sequences, input of 
digitized audio from an analog microphone or line 
source, etc. The application controls all this data 
flow in the context of a real-time clock: certain 



events under program control must occur at explic- 
itly defined points in time, and these times are 
defined very accurately (e.g., in milliseconds). 

According to the present invention, there is 
s now provided a multimedia data processing system 
comprising a personal computer having a memory 
for storing at least one multimedia application pro- 
gram and a multitasking operating system, data 
streaming apparatus operable under said operating 
10 system for streaming data from a source device to 
a target device, said apparatus comprising first 
means adapted, in response to execution of a 
stream create instruction in said application pro- 
gram, to create a source thread and a target thread 
75 as multitask threads under said operating system, 
to create a stream, and to allocate a plurality of 
buffers in said stream; second means operative in 
response to a start instruction in said application 
program, to first run said source thread and fill a 
20 plurality of buffers with data from said source de- 
vice, to then run said target thread and read data 
from said buffers into said target device, and to 
thereafter runs both threads by alternately filling 
and reading said buffers until an end of data 
25 stream is reached. 

The invention will be apparent from the follow- 
ing description taken in connection with the accom- 
panying drawings wherein: 

Fig. 1 is a block diagram of a data processing 
30 system embodying the invention; 

Fig. 2 is a block diagram of sync/stream sub- 
system architecture embodied in the system 
shown in Fig. 1; 

Fig. 3 is a block diagram illustrating a general- 
35 ized model of data streaming; 

Fig. 4 is a flow chart of source stream handler 
operations: 

Fig. 5 is a flow chart of target stream handler 
operations: 

40 Fig. 6 is a flow chart of another type of target 
stream handler operations; 
Fig 7 is a diagram illustrating unified and split 
data buffering; 

Fig. 8 is a diagram of buffering data structures; 
45 Figs. 9-19 are flow charts of interlocking 
sync/stream buffer management operations; 
Fig. 20 is a flow chart of sync/stream manager 
create stream operations; 

Fig. 21 is a flow chart of sync/stream manager 
so start stream operations; 

Fig. 22 is a diagram of event data structures; 
Fig. 23 is a flow chart of event enabling oper- 
ations; 

Fig. 24 is a flow chart how operations proceed 
55 from an application program to the sync/stream 
manager; and 

Fig. 25 is a flow chart of an example of event 
detection operations. 
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Detailed Description 

MULTIMEDIA SYSTEM 

The real-time, continuous, high-volume data 5 
transport requirements of a multimedia system 
present a set of interdependent problems that a 
generalized transport mechanism must solve: 

1. Throughput-intensive applications with very 
heavy data i/O demands - Uncompressed, digi- 10 
tal motion video at 640x480x16 resolution re- 
quires approximately 3 Mb/sec. Even highly 
compressed, a digital motion video data stream 

can require transferring 60 Kb/sec. Adding 8 or 
16-bit digital audio waveform data transfer into 75 
the scenario can increase the throughput load to 
80 Kb/sec or greater. Supporting this throughput 
load continually while the multitasking system 
also performs other application functions such 
as file I/O and keyboard/mouse device control is 20 
the most important requirement for a general- 
ized data streaming mechanism. But because 
the multimedia data presentations at the user 
interface (auditory and visual) are real-time de- 
pendant, these data transfers must occur within 25 
very tight real-time constraints. 

2. Control of multiple different hardware I/O de- 
* vices - Multimedia applications typically control 

a variety of I/O devices to manage presentations 
in the audio and video domains of the user 30 
interface. Audio devices that support waveform 
capture and playback, digital audio compact 
disc playback, Musical Instrument Digital Inter- 
face (MIDI) I/O, and voice-generation may be 
involved. Similarly, video devices that support 35 
NTSC (or PAL) video digitization, 
compression/decompression, and capture are 
routinely involved in multimedia presentation 
and/or authoring scenarios. The generalized data 
streaming mechanism must support these and 40 
future multimedia devices in a flexible manner, 
without requiring major modifications of the mul- 
timedia system extensions. 

3. Provide consistent control services across 
devices and data types - The data streaming 45 
mechanism must be controlled through services 

that are consistent across the different devices 
and data types involved in multimedia authoring 
and presentation scenarios. When new data 
types are introduced, as multimedia data stan- so 
dards evolve, this generalized streaming mecha- 
nism must not be substantially impacted. The 
control services must provide for easily incor- 
porating new data types into existing applica- 
tions. Likewise, these services must allow for 55 
seamlessly adding new I/O device types and 
capabilities, without impacting existing applica- 
tions or requiring major revisions of the mul- 



timedia extensions. 

4. Provide identical control services at multiple 
system privilege levels - Multimedia data ob- 
jects may originate (or terminate) at a hardware 
I/O device, or in system (or application) mem- 
ory. Where data streaming originates or termi- 
nates with a hardware device, the transport 
mechanism must be able to control behavior of 
the hardware device indirectly, rather than taking 
on direct control of the device, and thereby 
becoming device-specific itself. However, the 
nature of operating systems that support mul- 
tiple privilege (or protection) levels—a common 
practice in modern operating systems-requires 
that for optimal performance the transport 
mechanism ("stream handler") should be at- 
tached to the device's native physical device 
driver at the highest system privilege level. This 
is the execution level where hardware device 
drivers typically run. Although it is desirable, for 
performance reasons, to create device driver 
level stream handlers, the stream handler device 
driver must not become hardware dependent 
itself. All hardware-specific code must be encap- 
sulated in the device's native physical device 
driver. Alternatively, where no physical device is 
involved as the source or destination of the data 
stream, the stream handler may be implement- 
ed at a lower privilege level and still achieve 
optimal performance. Because the data transport 
mechanism is thus distributed across multiple 
privilege layers, identical stream control services 
must be provided at any level where the stream 
handlers execute. Multiple entry points at dif- 
ferent privilege levels should be provided with- 
out incorporating redundant code. 

5. Ability to extend to new devices/data types 
with minimal impact - New multimedia data 
standards and device types are being created 
almost continually. As this situation will persist 
for the indefinite future, the data transport 
mechanism must be able to easily and seamles- 
sly adapt to support the new devices and data 
types. 

6. Requirements for physical memory during 
streaming must be kept to a minimum - Despite 
the heavy throughput load imposed by. mul- 
timedia applications, there should be conserva- 
tion of memory resources at all times when data 
streaming is underway. Buffer management ser- 
vices should be provided so that applications do 
not need to become physical memory man- 
agers. However, since some applications create 
multimedia data objects in memory, these ap- 
plications may need to have direct control over 
the memory buffers. This capability should be 
seamlessly supported by the data streaming 
mechanism and its control services. 
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7. CPU loading must be kept to a minimum 
during streaming - Just as memory is a scarce 
resource that must be carefully managed, CPU 
cycles are another critical resource that cannot 
be squandered during multimedia application 
execution. The data streaming mechanism must 
keep CPU loads to a minimum by exploiting any 
direct hardware-to-hardware data transport capa- 
bilities (e.g., bus master data transfers). In addi- 
tion, interrupt-d riven device I/O should be ex- 
ploited wherever possible to reduce CPU load, 
rather than allowing the data transport mecha- 
nism to control devices using device-polling 
techniques. 

The following definitions are provided to estab- 
lish a framework within which the present invention 
can be described: 

Data Stream - A software means for data trans- 
fer through a data channel, in a continuous manner, 
from a source device (or memory address) to a 
target device (or memory address). 

Dynamic Link Library (DLL) - In the OS/2 ar- 
chitecture, a 16-bit or 32-bit executable module 
that executes at privilege levels 2 or 3, and is 
loaded into memory in order to resolve the external 
references of another executable (e.g., .EXE) mod- 
ule. A DLL can execute only in a task context. 

Physical Device Driver (PDD) - In the OS/2 
architecture, a 16-bit executable module which ex- 
ecutes at privilege level 0 (ring 0) and typically 
controls the operations of an I/O device. A PDD 
can execute in either a task context or a hardware 
interrupt context. 

Stream - To continuously transfer data from a 
source to a destination (or target). 

The invention provides an improved, general- 
ized, data streaming, multimedia data transport 
mechanism between various I/O devices, applica- 
tion memory, and system memory that is designed 
to solve the problems and satisfy the requirements 
noted above. The transport mechanism is em- 
bodied within extensions to a multitasking operating 
system which incorporates multiple execution privi- 
lege levels. The mechanism supports continuous, 
real-time, data transfer referred to as data stream- 
ing. 

Referring now to the drawings, and first to Fig. 
1, there is shown an exemplary data processing 
system comprising a personal computer 10 op- 
erable under a multitasking operating system such 
as OS/2 Version 2.0, to execute application pro- 
grams. Computer 10 comprises a microprocessor 
12 connected to a local bus 1.4 which, in turn, is 
connected to a bus interface controller (BIC) 16, a 
math coprocessor 18, and a small computer sys- 
tem interface (SCSI) adapter 20. Microprocessor 12 
is preferably one of the . family of 80xxx micropro- 
cessors, such as an 80386 or a 80486 micropro- 



cessor, and local bus 14 includes conventional 
data, address, and control lines conforming to the 
architecture of such processor. Adapter 20 is also 
connected to a SCSI bus 22 which is connected to 
5 a SCSI hard drive (HD) 24 designated as the 
C:drive, the bus also being connectable to other 
SCSI devices (not shown). Adapter 20 is also con- 
nected to a NVRAM 30 and to a read only memory 
(ROM) 32. 

70 BIC 16 performs two primary functions, one 

being that of a memory controller for accessing a 
main memory 36 and a ROM 38. Main memory 36 
is a dynamic random access memory (RAM) that 
stores data and programs for execution by micro- 

75 processor 12 and math coprocessor 18. ROM 38 
stores a POST program 40 and a BIOS 42. POST 
program 40 performs a standard power-on, self-test 
of the system when computer 10 is started by 
turning the power on or by a keyboard reset. An 

20 address and control bus 37 connects BIC 16 with 
memory 36 and ROM 38. A data bus 39 connects 
memory 36 and ROM 38 with a data buffer 41 that 
is further connected to data bus 14D of bus 14: 
Control lines 45 interconnect BIC 16 and data buff- 

25 er 41 . 

The other primary function of BIC 16 is to 
interface between bus 14 and an I/O bus 44 de- 
signed in conformance with Micro Channel (MC) 
architecture. Bus 44 is further connected to an 

30 input/output controller (IOC) 46, a video signal pro- 
cessor (VSP) 48, a digital signal processor (DSP) 
49, and a plurality of expansion connectors (EC) or 
slots 50. VSP 48 is further connected to a video 
RAM (VRAM) 60 and a multiplexor (MUX) 62. 

35 VRAM 60 stores text and graphic information for 
controlling what appears on the screen of a monitor 
68. MUX 62 is further connected to a digital to 
analog converter (DAC) 68 and to a connector or 
terminal 70 that is connectable to a video feature 

40 bus (VFB). DAC 66 is connected to monitor 68 that 
provides a conventional output screen or display 
for viewing by a user. 

IOC 46 controls operation of plurality of I/O 
devices including a floppy disc drive 72 designated 

45 as the A:drive, a printer 74, and a keyboard 76. 
Drive 72 comprises a controller (not shown) and a 
removable floppy disc or diskette 73. IOC 46 also 
is connected to a mouse connector 78, a serial port 
connector 80, and a speaker connector 82 which 

so allow various optional devices to be connected into 
the system. 

DSP 49 is further connected to an instruction 
RAM 84, a data RAM 96, an analog interface 
controller (AIC) 88, and an audio controller (90). 
55 RAMS 84 and 86 respectively hold instructions and 
data used by DSP 49 for processing signals. Audio 
controller 90 controls various audio inputs and out- 
puts and is connected to a plurality of connectors 
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92 by which various devices can be connected to 
the system. Such devices include a headphone, a 
microphone, a speaker, a musical instrument 
digitizing interface (MIDI), and devices requiring an 
audio line-in and line-out functions. Various other 
multimedia devices (MMD) 96 can be also attached 
to the system through an EC 50 and adapter card 
94. 

Memory 36 stores various, programs for execu- 
tion in the system, which programs include applica- 
tion programs 100, including multimedia application 
programs (MMAP) 102, and an operating system 
98 having extensions thereto including a 
sync/stream sub-system (S/SS) 104. It is to be 
noted that while Fig. 1 illustrates an exemplary 
multimedia system, the operating system is gen- 
eral purpose and is designed to run or control data 
processing systems having configurations that are 
different from the one shown in Fig. 1. The inven- 
tion is implemented using S/SS 104 and its inter- 
action with operating system 100, which will now 
be described. 

SYNC/STREAM SUB-SYSTEM 

Referring to Fig. 2, multimedia application pro- 
grams 102 execute at a layer above operating 
system 98 and communicate through multimedia 
control interface (MCI) by sending MCI commands 
for controlling devices in the multimedia environ- 
ment. Some of the basic commands are pause, 
play, record, resume, seek, save, set, stop, etc. 
Such commands are routed by the operating sys- 
tem 98 to a media device manager (MDM) 106. 
The application programming model for MMAPs is 
a logical extension of the OS/2 Presentation Man- 
ager programming model, incorporating both object 
oriented messaging constructs and procedural (call 
and return) programming interfaces. The MCI pro- 
vides a view to application developers and users 
similar to that of a video and audio home entertain- 
ment system. Operations are performed by control- 
ling, the processors of media information known as 
media devices. Media devices can be internal or 
external hardware devices, or there can be soft- 
ware libraries that perform a defined set of oper- 
ations by manipulating lower-level hardware com- 
ponents and system software functions. Multiple 
media devices may be included in . a scenario, and 
they can be allocated and controlled as a group for 
the purpose of synchronized playback. 

Multimedia applications must control two as- 
pects of real time system behavior, the transfer of 
large amounts of data from one device to another 
and the synchronization of events that are related. 
Events under the control of the program must be 
perceived to be under the direct control of the 
user, and the underlying system functions facilitate 
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and ensure that these events occur in a predict- 
able, real-time manner. Multimedia application au- 
thors should write programs that operate on a real- 
time clock basis, rather than an approximate clock 
5 that could allow events to occur within windows of 
probability. 

The MCI has two levels of dynamic link librar- 
ies (DLLs) comprising MDM 106 and media drivers 
including an audio media driver 108 and a MIDI 
70 media driver 110. MDM 106 provides resource 
management for media devices. It resolves conten- 
tion for access to media devices and provides the 
application developer a view of resources that is 
independent of hardware. The media drivers are 
75 dynamic link libraries that implement the functional- 
ity of media devices. Media drivers invoke the 
services of hardware devices or software to imple- 
ment their functionality. The media drivers do not 
directly control the hardware devices. Instead, they 
20 pass commands to S/SS 104 through a stream 
programming interface (SPI) 112 to a sync/stream 
manager (SSM) 114 which controls synchronization 
and streaming activities. SSM 114 performs two 
primary functions, the first being to control data 
25 streaming to provide continuous, real-time data 
streaming. The second primary function involves 
synchronizing data streams. 

Stream handlers are required at both the sys- 
tem kernel level and the application level. Some 
30 data streams are best controlled by a direct con- 
nection between a stream handler and the physical 
device driver at the Ring 0 priority level Such 
stream handler communicates with the PDD using 
a common interface based on OS/2 Interdevice 
35 Driver Communication (IDC). Other data streams 
are not associated with a data source or target that 
can be mapped to a specific physical device and 
can be controlled at the Ring 3 priority level by a 
DLL. Dotted line 113 generally indicates which 
40 items operate at the different priority levels. Within 
SSM 114, some routines operate at one level and 
other routines operate at the other level, as appro- 
priate to the task at hand. 

Each stream handler is programmable and is 
45 capable of streaming according to stream proto- 
. cols. From the perspective of SSM 114, all stream 
handlers have similar responsibilities. Each handler 
is designed to be the source or target for one or 
more data streams where each data stream moves 
so data independently, Manager 114 is responsible for 
connecting a stream' source to an appropriate 
stream target, for maintaining data flow, and clean- 
ing up the various resources when the stream has 
ended. Further, the stream handlers are not device 
55 dependent modules. Although each stream handler 
supports streaming data of specific predefined 
types, data is passed from one handler to the next 
without any knowledge of hardware behavior. Also, 

5 
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audio stream handler 134 can communicate with 
any compatible audio device PDD in a completely 
hardware independent manner. To be compatible, 
the PDD must conform to the IDC interface as well 
as to the standard audio device driver interface 
IOCTL. Thus as shown, stream manager 114 inter- 
acts with a plurality of stream handler dynamic link 
libraries (DLL) 116-126 which respectively are MIDI 
mapper, CD/DA, memory, split stream, CD- 
ROM/XA, and file system, stream handlers. File 
system stream handler DLL 128 calls a multimedia 
I/O (MMIO) manager for accessing a file system 
130. 

Stream manager 114 also interacts through 
stream manager helpers 133 with an audio stream 
handler physical device driver (PDD) 134 that se- 
lectively accesses physical devices through a 
stream handler/ PDD interface 136 and a plurality 
of PDDs 138-144. Stream manager 114 can also 
interact with interface 136 through inter-device 
communication (IDC) interfaces 146: 

Fig. 3 is a generalized model of data streaming 
operations the details of which are discussed rela- 
tive to the flow charts and data structures in the 
remaining figures. Fig. 3 shows generally a single 
data stream 151 and how data flows or is trans- 
ported under the control of stream manager 114, 
and source and target stream handlers 1 52 and 
154. A plurality of stream buffers 156 are allocated 
in memory for use in streaming. Buffers 156 are 
filled wjth stream data from a source device 158 
and are emptied of stream data by transmitting the 
data to a target device 160. Data stream 151 com- 
prises two portions, source stream data 153 and 
target stream data 155. The data path for the 
source stream data is from source 16, through 
source PDD 162, and through stream handler 152 
to buffers 1 56. The data path for the target stream 
data 155 is from buffers 156, through target stream 
handler 154, through target PDD 164, and to target 
device 160. Source stream handler 152 actuates a 
source PDD 162 which in turn controls operation of 
the source device. Target stream handler 154 ac- 
tuates a target PDD 164 which controls target de- 
vice 160. The general objective is for the source 
stream handler 152 to fill at least two of stream 
buffers 156 before the target device is started, and, 
once the target device has started, to thereafter 
keep ahead of the target device by filling buffers, 
until the complete source data has been trans- 
ferred. After the buffers ar filled, the target stream 
handler can then obtain target data therefrom and 
transfer it to the target device. 

Media driver 108 interacts with SPI interface 
112 by sending SPI functions or calls and receiving 
stream event notifications. Manager 114 interprets 
the SPI calls and in response thereto performs the 
desired functions by interacting with the stream 



handlers by sending system helper commands 
SHCs to the handlers and receiving stream man- 
ager helpers SMH calls from the handlers. Media 
driver 108 can also directly control PDD 164 by 

s issuing control functions defined by standard 
IOCTL commands. The principal SPI calls related 
to the invention are SpiCreateSream and SpiStart- 
Stream which respectively setup up the desired 
stream(s) and then start the data streaming, as 

w discussed in more detail below with reference to 
Figs. 20 and 21. Should there be plural streams 
that must be synchronized, a SpiEnableSync call is 
made, as more fully discussed in the related patent 
application. 

75 Fig. 4 illustrates what happens when a stream 

handier command SHC. CREATE is executed. How- 
ever, before describing that, it might be useful to 
understand how that point is reached relative to an 
application program. First the MMAP issues an 

20 MCI call to create a data stream. Such call is 
analyzed by the OS in 98 which sends information 
to media device manager 106. This manager then 
selects the appropriate media driver that sends an 
SpiCreateStream call to sync/stream manager 114 

25 which then issues the SHC. CREATE command to 
the appropriate stream handler. 

Fig. 4 illustrates what happens when the 
SHC. CREATE command is executed by a DLL 
source stream handier 152. First, step 201 creates 

30 a thread under the OS which will be controlled by 
the multitasking features of the OS. Step 202 as- 
signs a priority level to the thread as appropriate to 
the task to be performed, and then the thread is 
blocked on a semaphore in step 203 and goes to 

'35 "sleep". The priority lever may be used to control 
the rate at which the OS dispatches and executes 
the thread. If discontinuities arise, priority levels 
may be adjusted. 

It is to be noted that in accordance with the 

40 standard operation of OS/2, the threads are treated 
as individual tasks, and control returns to the OS 
when the threads are blocked, when calls are 
made, when returns are made, etc. This allows the 
operating system to execute other tasks in the 

45 system and to return to the streaming threads, on a 
multitasking basis. It should also be noted that 
several connectors appear in the drawings, the 
connectors including a circled numeral along with 
arrows indicating the direction of program flow. For 
so example, connector 1 appears in both Fig. 4 and 
* Fig. 20. In Fig. 20 the connector includes both call 
and return arrows, whereas the connector 1 in Fig. 
4 includes only a call arrow indicating the routine is 
being called elsewhere. The other connectors can 
55. be interpreted in the same manner. 

In response to the start command being made 
in the application program, manager 114 sends a 
SHC. START command first to the source thread 
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handler in 206 (Fig. 4) and then to the target 
stream handler. The source stream handler needs 
to be started first to fill stream buffers before the 
target handler can use the data being transferred 
thereto. In response to such command, the source 
stream handier in step 208 unblocks the source 
thread. In response to being unblocked or awak- 
ened, source thread then requests, in step 216, an 
empty buffer from manager 114. If a buffer is not 
available, as determined in step 218, step 220 then 
blocks the thread again. If an empty buffer is 
available, then step 222 reads data from the source 
device and fills the buffer. Step 224 then returns 
the filled buffer to manager 114. Step 226 decides 
if any more buffers need filling. If so, a branch is 
made back to step 216 and a loop is formed from 
steps 216-226 which loop is broken when step 226 
decides no more buffers need filling. Then, the 
thread is blocked. Once the streaming operation 
has been started, buffer filling process repeats until 
the end of the source file is reached at which point 
the source thread quiesces. 'Event detection can be 
inserted in the sequence of operation at any of the 
points indicated with an asterisk in Fig. 4, e.g., 
using cue point detection to keep track of time 
where time is determined by the rate of data trans- 
ferred. Event detection can also be used in many 
of the routines that follow. 

Fig. 5 illustrates operating the a DLL target 
stream handler at ring 3 priority level. Target 
stream handler receives from manager 114 an 
SHC. CREATE to also create a target data stream 
and it first creates a target stream thread in step 
230, assigns a priority to the thread in step 232, 
and blocks the target thread on a semaphore in 
step 234. When the target thread is awakened by 
manager 114 issuing a SHC. START command in 
step 236, the target thread is unblocked in step 
238 to awaken the thread in step 240. Step 242 
attempts to get a full buffer from the data stream. 
Step 244 checks to see if a buffer is obtained, and, 
if not, step 246 blocks the target thread. If a full 
buffer was obtained, step 248 writes the data in the 
buffer to the target device and then returns an 
empty buffer in step 250 to manager 114. Step 252 
checks to see if the buffer is the last buffer in the 
data stream i.e. has the EOS been reached. If not, 
a branch is made back to step 242 to repeat the 
process of getting a full buffer. The loop repeats 
until the end of stream is detected, whereupon step 
244 ends the thread. 

Fig. 6, to which reference is now directed, 
shows a target stream handler implemented in a 
physical device driver at a ring 0 priority level. 
When an SHC.CREATE command is received in 
step 256, step 258 sets up the stream data struc- 
tures and step 260 initializes the target device. A 
return is made in step 261. When the stream is 
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started in step 262, a request for a full buffer is 
made in step 264 to manager 1 1 4. Step 266 deter- 
mines if a buffer was given. If so, step 268 initiates 
a Write I/O and a return is made by step 270. If no 

5 buffer was returned as a result of step 272 deter- 
mining there was an error underrun, step 274 re- 
ports the error and then a return is made. 

When the I/O operation is completed by writing 
and emptying the buffer, step 272 then make an 

10 interrupt which is handled by an interrupt handler in 
step 274. If the I/O operation was done without 
error, step 278 then returns the empty buffer to 
manager 114, and a branch is made to step 264 to 
repeat the process. If an error occurred during the 

75 I/O operation, step 280 reports the error and a 
return is made in step 282. A source stream han- 
dler operating a Ring 0 priority level performs 
operations the same as shown in fig. 6 except that 
it gets an empty buffer and returns a full buffer. 

20 There are two types of data files and streams 

which can be used, a unified or non-interleaved 
stream and a split or interleaved stream. A unified 
data stream and file contains data of only one type', 
e.g., only audio data. A split stream contains data 

25 of more than one type, e.g, both audio data and 
video data. An example is the type of file com- 
monly created for CD-ROM devices. Fig. 7 shows 
examples illustrating the different types. File 286 
represents a unified file containing only audio data 

30 of a given type. Such file is read info a plurality of 
buffers B1-B3, by filling buffer B1 with a first sec- 
tion of the audio data, then B2 is filled with the next 
section of audio data, etc. File 288 is a split or 
interleaved file having a plurality of sections of 

35 interleaved audio and video data. The sections are 
paired so that the audio corresponds to the video 
being presented. When such data is transferred 
into associated buffers B4-B6, it is done so as 
records. Thus, the first audio section A1 becomes 

40 audio record AR1 in B4, the first video section V1 
becomes video record VR1, in B4, and the second 
audio section A@ becomes AR2 in B4. Next, sec- 
tions V2, A3, and V3 are transferred to buffer B5 as 
records VR2, AR3, and VR3. Buffer B6 is similarly 

45 filled with AR4 and VR4. Quite obviously, in this 
. example, the buffers B4-B6 are each of a size to 
hold three records. 

When streams are created by manager 114, a 
plurality of buffers are allocated to each stream 

so and, data structures are formed containing data 
used to manage the streams. Referring to Fig. 8, 
for each stream, a buffer directory structure BDS is 
created which is added to a linked list of BDSs of 
other streams. A chain of buffer control blocks is 

55 created for each stream, there being one BCB for 
each buffer allocated to the stream. As shown, 
BDS1 is created for stream #1 is chained to three 
RCBs RCB1-RCB3 which respectively are asso- 
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ciated with and point to three buffers B1-B3 al- 
located to stream #1 . 

. Each BDS contains a plurality of fields for 
storing information on the number of buffers , al- 
located to the stream, the state of the stream, the 
number of records in a buffer (for split streams), a 
pointer 200 to the RCB of the first full buffer, a 
pointer 202 to the RCB of an empty buffer, and a 
pointer 204 to the next BDS in the list. The dif- 
ferent states of a stream include, running, stopped, 
paused, and prerolled or primed. The latter is an 
option which allows for quick starts by filling all of 
the buffers from the source . prior to starting the 
target device. Each BCB stores information includ- 
ing the number of bytes in the buffer, the number 
of records in a split stream buffer, a pointer 206 to 
the buffer, and pointer 208 to the next BCB in the 
chain, and pointers to the head and tail of the RCB 
chain. 

Streams #3 and #4 illustrate buffering data 
structures for split streams using the example in 
Fig. 7. A BDS is set up for each data type in the 
split stream. BDS3 is setup for audio data , and 
BDS4 is set up for the video data. As buffer B4 is 
filled with the three records, two record control 
blocks RCB1 and RCB2 are created and filled to 
indicate the offsets into B1 of audio records AR1 
and AR2, and the lengths thereof. BCB4 is set to 
indicated there are two records in buffer B1. Con- 
currently, RCB3 is also created to indicated the- 
offset into buffer B1 of video record VR1 and the 
length thereof. The number of records field in 
BCB7 is set to one corresponding to only one 
video record being stored therein. Additional BCBs 
and RCBs are similarly created and filled for the . 
remaining buffers allocated to the split stream. Also 
for a split stream, one BDS is designated owner 
and any remaining BDS is considered a user. The 
linked lists contains pointers 210 from each user 
BDS to the associated owner BDS. Further there 
are the following pointers: a pointer 212 from a 
user BCB to the owner BCB; and pointers 214 from 
a BCB to RCBs chained thereto. 

SSM 114 is called from the stream handlers 
using stream manager helpers (SMHs). Referring to 
Fig. 9, when manager 114 receives a SMH. NOTIFY 
call at 220 to get or return a buffer or a record, the 
call parameters are verified in step 222, and step 
224 determines whether the caller is a source 
stream handler or a target stream handler. If it is a 
source system handler, step 226 determines if the 
user or application is supplying buffers. If so, step 
228 calls the GiveBuf routine and upon being re- 
turned to in step 228, step 230 returns to the caller. 
If step 226 results in a negative determination, step 
232 decides if the a full buffer is being returned. If 
so, step 234 then calls ReturnFull routine and upon 
its return, a return is made through step 230 when 
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a full buffer is returned or step 236 is executed to 
determine if the request is for an empty buffer. 
Step 236 is also executed when step 232 deter- 
mines the request is not for a full buffer. If 236 
5 makes a positive determination, step 238 then calls 
the GetEmpty routine and returns via 230. 

If step 224 indicates the caller is not a source 
stream handler, meaning it must be a target one, 
step 240 determines if an empty buffer is being 
70 returned. If so, step 242 the calls ReturnEmpty 
routine and step 244 determines if the request is 
for a full buffer. If not, a return is made via 230. If 
so, step 246 calls the GetFull routine before return- 
ing via 230. If step 240 results in a negative deter- 
75 mination, step 248 checks to see if the end of the 
stream has , been reached. If not, step 244 pro- 
ceeds as before. If so, step 250 resets the buffers, 
and step 252 reports an end of stream event to the 
application before passing to step 244: 
20 Fig. 10 shows the GiveBuf routine 228 in which 

step 254 checks to see if there is any remaining 
buffer to give. If not, a return is made in step 256. 
If there is-such a buffer, step 257 checks to see if 
an empty pointer points to an empty BCB. If one is 
25 pointed to, step 258 calls MapUserBuffer routine 
258. Afterward, step 260 checks for errors. If none 
occurred, step 262 saves buffer attributes and step 
264 checks to see if the end of the stream has 
been reached. If not, step 266 checks to see if a 
30 Target.is.waiting flag is set. If not, a loop is made 
back to step 254. If step 256 produces a "no" 
answer, step 268 sets Source.is.waiting flag on and 
step 270 calls a CheckPreroll routine. Step 272 
checks for erron If none, control passes to step 
35 254. If an error is noted in 272, step 274 sets error 
flag indicating too many buffers and a return is 
made by 256. If step 264 answers positive, step 
276 calls check-preroll routine and then goes to 
step 254. If step 266 is positive, step 278 sends a 
40 helper command to start the target stream handler. 
Referring to Fig. 11, when the ReturnFull rou- 
tine is called to obtain a full, buffer, step 280 first 
checks to see if there are any full buffers available. 
If none is, a return 282 is made. Is- there is a full 
45 buffer, step 284 searches the chain of BCBs for the 
full buffer address and step 286 checks if a full 
buffer is found and if the source owns it. If not, 
step 288 sets error flag to indicate invalid buffer 
returned, and then return 282 is made. If step 286 
so is positive, step 290 determines if a buffer or a 
record is being returned. If it is a buffer, step 292 
saves the buffer attributes and step 294 checks to 
see if there is a pending discard stop. If so, step 
300 calls the CheckStopPending routine. Step 296 
55 follows 300 or 294 and determines if the end of 
stream has been reached. If so, step 302 calls the 
CheckPreroll routine. If the end of stream is 
reached, step 298 checks to see if the target 
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stream handler is waiting to be started. If so, step 
299 calls the SHC. START routine to start target. If 
step 298 is negative, a loop to step 280 is made. 

If step 290 determines a record is to be re- 
turned, step 304 validates that the record is in the 
buffer. Step 306 determines if the record pointer is 
ok. If not, step 308 sets an error flag indicating an 
invalid record has been returned, and step 282 
then returns. If step 306 is positive, step 310 saves 
record attributes, allocates an RCB and adds it to 
BCB list. Then, step 312 checks for end of stream 
and last record for the buffer conditions, and step 
294 follows. 

At the start of the GetEmpty routine shown in 
Fig. 12, step 314 determines if there are more 
buffers to get. If not, a return is made in 316. If so, 
step 318 checks if an empty pointer points to an 
empty BCB. If so, step 320 gets the buffer at- 
tributes and the loops back to 314. If step 318 is 
negative, step 322 turns on a Source.is.waiting flag 
indicating no buffer is available. Step 324 then calls 
the CheckPreroll routine. Step 326 checks for error. 
If none, a loop to step 314 is made. If there is an 
error, an error flag is set to indicate no buffer is 
available and a return is made by 316. 

Referring to Fig. 13, ReturnEmpty routine be- 
gins with step 330 which checks to see if there are 
more buffers to return. If not, step 332 returns. If 
there are buffers, step 334 searches the chain of 
buffer BCBs for buffer address. Step 336 sees if 
buffer is found and if the target stream handler 
owns it. If not step sets an error flag indicating an 
invalid buffer is returned. If step 336 is positive, 
step 340 determines if a record or a buffer is being 
returned. If it is a buffer, step 342 marks the buffer 
as empty, and step 346 sees if the source stream 
handler is waiting, if not, step 348 is skipped. If so, 
step 348 calls the source SHC. START routine. If 
step 344 is positive, step 350 then calls the Check- 
StopPending routine. Step 330 then follows either 
step 348 or 350. If a record is returned as deter- 
mined in 340, step 352 removes the associated 
RCB from the RCB list and check 354 determines 
if all records have been returned empty. If so, step 
356 marks the buffer as empty and step 344 then 
follows either step 354 or 356. 

In the GetFull routine (Fig. 14), step 358 deter- 
mines if there are more buffers to get. If not, return 
360 occurs. If so, step 362 sees if the full pointer 
points to a full buffer. If not, no buffer is available, 
and step 370 sets the Target.waiting flag on and 
returns via 3360. If 362 is positive, step 364 deter- 
mines the type of stream , normal (unified) or 
interleaved. If unified, step 366 gets the buffer 
attributes, step 368 sets the end of stream con- 
dition (EOS) when it is last buffer, and then returns 
via 360. For an interleaved or split stream, step 372 
searches for a full record in the buffer. If one is 
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found, as checked by 374, step 376 gets the 
record attributes and step 378 checks for last 
record in buffer before returning via 360. 

Referring to Fig. 15, the CheckStopPending 
5 routine checks in step 380 to see if all buffers have 
been returned. If not, a return is made by step 384. 
If so, step 386 calls the routine SMH. Re port Event 
to report "Stream Stopped", and then returns by 
384. 

10 Fig. 16 shows the operation of the 

CHECK. PREROLL routine in which step 388 deter- 
mines if the stream has been prerolled or primed 
and it is not in a sync group. If it is not, step 390 
checks if the stream a master and all slaves have 

75 been prerolled. If not, a return is made via 392. If 
the results of steps 388 and 390 are positive, step 
394 calls SMH.ReportEvent routine to report the 
"Stream Prerolled" before returning via 392. 

Fig. 17 shows the SMH.ReportEvent routine 

20 which is a call available at Ring 0 and Ring 3. Step 
396 checks an event queue to see if it has over- 
flowed. If it has, step 397 reports an error event 
and that "Event Queue Overflow" and then contin- 
ues with step 398. Also, if there is no event queue 

25 overflow, step 398 adds the event to the event 
queue for the specific process, step 400 unblocks 
the event thread at ring 3, and step 402 returns. 

The operations in the MapUserBuffer routine 
are shown in Fig. 18 in which step 404 searches to 

30 see if buffer is already locked, i.e.- buffer already in 
own list. If the buffer is locked, step 406 goes to 
step 416 which copies the BCB for the buffer to a 
new BCB list and then returns by 418. If the buffer 
is not locked, step 408 tries to lock the buffer. Step 

35 410 determines if latter step was successful. If so, 
step 420 adds field information to the BCB and 
adds BCB to tail of buffer queue before returning at 
418. If 410 is not successful, step 412 allocates a 
new locked buffer, and step 41 4 copies buffer data 

40 to new buffer, before passing on to steps 420 and 
418. ^ 

As shown in Fig. 19, event checking is ob- 
tained be adding to the routine at the desired point, 
a step 422 which blocks the thread. When the 

45 thread becomes unblocked, the EventRouter rou- 
tine is called in step 424 which first checks in step 
426, the event queue for an event and then de- 
queues the event in step 428. Step 430 calls the 
Application Event routine which handles the event. 

so Step 432 then sees if there are any more events to 
process and if so, returns to step 428. When all 
events are processed, step 434 then blocks the 
thread. 

As shown in Fig. 20, when an application pro- 
55 gram creates a stream, an SpiCreateStream API 
call is made to pass control to sync/stream man- 
ager 114 which in step 436 calls the source stream 
handler using the command SHC. CREATE. Then, 
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the target stream handler is created using 
SHC. CREATE in step 438. Step 440 obtains the 
streaming parameters which are listed in a stream- 
ing protocol control block (not shown) and includes 
buffer numbers and size, data stream type, maxi- 5 
mum number of records per buffer, minimum num- 
ber of buffer needed to maintain constant data 
stream, number of empty buffers, needed to start 
source stream handler, number of full buffers need- 
ed to start- target stream handler, etc. Then, step w 
442 informs the source stream handier of the 
stream parameters, step 444 informs the target 
stream handler of the stream parameters, step 446 
sets up the stream and allocates the stream buff- 
ers, and step 448 returns to the application. 75 

Referring to Fig. 21, when an application pro- 
gram issues a start streaming call, manager 114 
receives an SpiStartStream API call, and step 450 
executes the SpiStartStream routine which first 
checks in step 452 to see if the stream is paused. 20 
If not, step 454 determines if the stream is prerol- 
led. If not, step 456 calls the source stream handler 
with the command SHC.START to unblock the 
source thread. Then step 458 checks to see if the 
stream is a slave stream in a sync group. If not, 25 
step 460 returns to the application program. If so, 
step 462 branches back to step 452 to repeat 
process for each slave stream. If step 452 answers 
"yes", step 464 calls the source stream handler 
with a SHC.START command to unblock the thread 30 
and then makes a call in step 466 to unblock the 
target stream handler. Step 466 also follows a 
"yes" result from 454. Step 458 then follows step 
466. 

Fig. 22 illustrates the event data structures. For 35 
each process, a process control block PCS is cre- 
ated which points to an event queue. AS illustrated; 
there are two PCBs, PCB1 and PCB2 respectively 
set up for processes #1 and #2. Each PCB con- 
tains a field for the process ID, a field for an Event 40 
Thread ID, and. a pointer 472 to the head of the 
associated event queue. The event queue contains 
a chain of headers 474 each of which contains a 
flag and a pointer to a users event control block 
EVCB 478. Such block, as shown in the enlarge- 45 
ment, specifies the event type, event flag, stream 
handler ID, its status, and user parameters. The 
information in the event data structure is used by 
the event router routine (Fig. 19). 

Fig. 23 illustrates operations of an SpiEnable so 
routine 480 in which step 482 determines the 
stream handler to call, and step 484 calls the 
source or target handler identified in step 484. Step 
484 is followed by a return 486. 

Fig. 24 illustrates what happens with the first 55 
Spi API call made at DLL initialize time, from an 
application. Step 488 creates and event thread by 
calling the EventRouter routine. Step 490 sets the 



priority of the event thread (ring 3 or 0), step 492 
calls the sync/stream manager, and step 494 re- 
turns. 

Fig. 25 provides an example of an event called 
a time cue point event, which can be placed in a 
stream handler, most likely a target stream handler. 
Step 496 determines the current time. Step 498 
then searches the event queues or list for any 
events which are to occur at such time. Step 500 
checks for any time matches. If there are none, 
step 504 returns. For each match, step 502 reports 
the event by call SMH.REPORTEVENT. 

Summary Description 

There have been described several unique fea- 
tures that are believed novel and have not pre- 
viously been integrated for multimedia data trans- 
port in any application or operating system soft- 
ware, the unique features including the following 
characteristics: Central Buffer Management with 
Application Buffering Option; Supports Bi-Level 
Data Stream Handlers; Support for Interleaved Data 
Streams; and Data Stream Event Detection and 
Notification. These unique features are implement- 
ed in a fully device-independent manner (beyond 
any existing physical device drivers). This solution 
further imposes no data type dependence on mod- 
ules within the multimedia system extensions. 

The generalized data streaming mechanism 
provides for device-independent multimedia data 
transport and is designed as multimedia extensions 
to IBM Operating System/2 (OS/2) Version 2.0. The 
mechanism provides system services that support 
the continuous movement of data for storage 
and/or retrieval, within real time constraints. For 
example, large digitized audio data files must be 
read from a hardfile and "played" by continuously 
transferring the data to a Digital-to-Analog Con- 
verter (DAC) device equipped with analog audio 
amplification circuitry. 

The following four major features represent a 
significant contribution to multimedia computing 
systems. 

Central Buffer Management with Application 
Buffering Option 

Data stream resource management controls 
buffer allocation to and frorn a central buffer pool, 
sharing buffers among multiple stream handlers, 
and dynamic buffer locking/unlocking. In addition, 
data streams may use buffers allocated and man- 
aged by applications, particularly where the ap- 
plication is controlling the real-time creation of data 
to be transferred via the stream. 

A set of buffer management routines provide 
the ability to get and return buffers from a pool of 
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buffers allocated when the data stream is created. 
The memory for these buffers is allocated and also 
locked down at this time. The need to lock down 
the buffers ensures the system will not swap out 
the memory containing the buffers while the data is 
streaming. 

To ensure the data being processed will 
stream in a continuous manner, a stream protocol 
control block associated with the stream is set up 
to contain information used to allocate the buffers, 
(i.e., the number of buffers, the buffer size, and 
values indicating when to start the stream han- 
dlers). This information is important because de- 
pendent on the type of data being streamed, the 
amount and size of . buffers must be optimum to 
ensure that continuous streaming of data will occur, 
and at the same time, ensure that the system 
resources are not over used, (i.e., allocate more 
memory than is actually needed). Along with the 
allocation of memory, the values used to start the 
stream handlers ensures that enough buffers are 
filled by the source stream handler that the target 
stream handler can be started and assured that 
there is enough data buffered up to allow real-time 
data streaming to occur. 

The Central Buffer Management routines allows 
for the source stream handler to get an empty 
buffer from the buffer pool, fill it with data from the 
input device (e.g., data from a file on the hard 
disk), and return the full buffer back to the buffer 
pool. The target stream handler will get a full buffer 
from the buffer pool, send this buffer to the output 
device (e.g^, data sent to the audio device to be 
played to the speakers), and then return the empty 
buffer back to the buffer pool. 

One unique feature of the buffer management 
routines is the ability to handle records within a 
buffer. This ability allows interleaved data to be 
processed with the minimum amount of data move- 
ment. The data is moved only once to the buffers, 
and subsequent access is via records within the 
buffer that contains the individual data (i.e., audio 
and video data etc.). 

Another unique feature of the buffer manage- 
ment routines is the ability to allow a source stream 
handler to supply its own buffers to be added to 
the buffer pool and subsequently given to the tar- 
get stream handler on request. As the buffers are 
given, they are locked down in memory at that time 
assuring they won't get swapped out and at the 
same time not consuming all the system memory 
resources for the entire duration of the data stream. 

The following is a typical scenario that a typical 
stream handler would use to read data from a file 
and stream to a audio output device: 

1. Application creates the data stream. The buff- 
ers will be allocated and locked down at this 
time. 
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2. Application will start the data streaming. 

3. Source stream handler will: 

- Get an empty buffer. 

- Fill the buffer from the input device. 
5 - Return the full buffer. 

- Loop on the above three steps until the 
end of the data is encountered. 

4. Target stream handler will: 

- Get a full buffer. 

io - Drain the buffer to an output device. 

- Return the empty buffer. 

- Loop on the above three steps until the 
end of the data is encountered. 

This set of buffer management routines pro- 
15 vides the capability to ensure a continuous flow of 
data can occur from the source stream handler to a 
target stream handler. 

Supports Bi-Level Data Stream Handlers 

20 

Provides identical system services for data 
stream handlers at different protection levels, allow- 
ing stream handlers to be developed as either 
OS/2 Physical Device Drivers (Protection Ring 0) or 

25 OS/2 Dynamic Link Libraries (Protection Ring 3). 

The OS/2 architecture supports the Intel family 
of processors based on the 80286, including the 
80386DX, 80386SX, and 80486, among others. 
These processors share a protection mechanism 

30 based on providing a series of privilege levels, or 
rings, numbered 0 to 3. At each level, different 
operations are permitted according to the relative 
risk that operation might pose to the integrity of the 
system. At privilege level 0, or "ring 0", all oper- 

35 ations are permitted. This level is typically reserved 
for system code execution. Applications run at ring 
3, where attempts to execute privileged instructions 
will generate a trap, handled by the operating sys- 
tem at ring 0, and typically resulting in the termina- 

40 tion of the application. 

Developing software at ring 0 is more difficult 
and more risky than at ring 3, due to the relative 
absence of system protection mechanisms active 
at ring 0. Special tools are required, including a 

45 Kernel debugger, in order to develop, test, and 
remove errors from ring 0 code modules. As a 
result, any function that does not require the spe- 
cial characteristics of ring 0 code, for instance the 
ability to execute in a hardware interrupt context, is 

so typically written as a ring 3 executable. Debugging 
at ring 3 is much easier, and any runtime errors 
generated by ring 3 code will generally have no 
impact on other processes due to the greater pro- 
tection afforded all ring 3 executables. 

55 Usually, developers prefer to develop all code 

as ring 3 executables, except in cases where the 
special characteristics of ring 0 privilege provide 
the desired function. However, in developing OS/2 
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ring 0 modules the number and flexibility of avail- 
able system services is quite limited. The only type 
of module that can be loaded for ring 0 execution 
is a Physical Device Driver (PDD). The only system 
services available for PDDs are the Device Help 
(DevHIp) functions provided by the OS/2 Kernel. 
These same DevHIp interfaces are not available to 
ring 3 modules, which must be programmed using 
a totally different set of system service interfaces. 

The problem may arise, then, whenever an 
extension of the operating system is introduced. If 
the new services are provided only at ring 3, then a 
wide range of device control capabilities are im- 
peded because modules which invoke the new 
services cannot also execute privilege level 0 
instructions or execute in a hardware interrupt con- 
text. If the new services are provided only at ring 0, 
then all developers will suffer from the difficulty of 
creating ring 0 modules, whether they need to 
access privilege level 0 instructions or not. 

The preferred embodiment is an OS/2 exten- 
sion which provides data stream control services. 
These services are exported by the Sync/Stream 
Manager which is comprised of an OS/22 DLL and 
PDD pair. A data stream is controlled by one or 
more Stream Handlers that invoke the services 
exported by the Sync/Stream Manager. 

In order to provide the greatest flexibility to 
stream handler developers, and to prevent rapid 
growth of ring 0 code, identical services are . ex- 
ported by the Sync/Stream Manager at both ring 0 
and ring 3. This means that only those stream 
handlers that absolutely require ring 0 privilege will 
be written as OS/2 PDDs, while all others may be 
written as OS/2 DLLs, running at ring 3. Because 
the same interfaces are available at both privilege 
levels, the stream handler developer can create 
code using only one set of interfaces for controlling 
data streams. 

The Sync/Stream Manager exports the same 
services to both ring 0 and ring 3 modules without 
creating duplicate service routines. Two entry 
points are exported for each function, one at ring 3 
(in the Sync/Stream Manager DLL) and the other at 
ring 0 (in the Sync/Stream Manager PDD). Each of 
these entry points then vectors into a common 
subroutine. The common code may execute jn 
either ring 0 or ring 3, whichever is most appro- 
priate to the specific function being invoked. 

Support for Interleaved Data Streams 

This mechanism allows a single data stream 
source to stream data to multiple targets. This is 
called a split stream. For example, a device might 
interleave audio, image, and text data types within, 
a single, formatted data track. A split stream han- 
dler is then used to extract each data type from the 



single interleaved data buffer, stream each data 
type independently, and deliver each stream to the 
appropriate target device. For example, CD-ROM 
XA can interleave waveaudio, video data and text 
s data within the same data buffer. The Split stream 
handler is used to "split" the data into multiple 
streams for CD-ROM XA devices. It is more effi- 
cient for the Split stream handler to read this 
combined data into one buffer and then insert the 
10 video data into a video stream and the waveaudio 
. data into an waveaudio stream without copying the 
data. The waveaudio data stream would then be 
consumed by an waveaudio stream handler and 
the video stream by a video stream handler. The 
75 waveaudio stream handler would "see" only the 
audio portions of the data stream and the video 
stream handler would "see" only the video portions 
of the stream. 

A data stream can be "split" into multiple Split 
20 streams. The number is limited only by the re- 
sources of the system. Each separate "split" of the 
stream is a separate stream. The work of parsing 
and determining the different data is the job of the 
source stream handler, which actually fills an emp- 
25 ty buffer with data from the source device. The 
source stream handler must determine which data 
goes in which stream. It must then return to the 
-pointers to the individual "records" of the buffer. 
Each record is a portion of the buffer filled with 
30 data specific to one of the "split" streams. These 
records are returned to the sync stream manager 
indicating which stream they should go to. The 
sync stream manager will then queue these 
records in the respective buffer/record queue for 
35 one of the split streams sharing the buffer. 

This type of split streams can be set up as 
follows. Multiple streams must be created, each 
with an Spi Create Stream api call to the sync 
stream manager. The first of these api calls is a 
40 normal stream creation call with a hstreamBuf pa- 
rameter set to NULL. Every other subsequent 
SpiCreateStream call that needs to share the buff- 
ers of the first stream must pass that streams 
handle in the hstreamBuf parameter. This is the 
45 mechanism used by the sync stream manager to 
allow these split streams to share the buffers of the 
first stream. The source stream handler will also 

receive this parameter on an SHC CREATE call 

and need to support this kind of streaming. A 

so SPCBBUF INTERLEAVED flag in the SPCB is set 

by the source stream handler to indicate whether it 
can do split streaming. If it can not, any 
SpiCreateStream attempting to use its buffers will 
be rejected. The split stream mechanism only 
55 works for one source and multiple targets. 
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Data Stream Event Detection and Notification 

Stream events identify a specific change of 
state in a data stream. The data stream handler 
detects the specific event, and may either ignore 
the event or post a notification that the event oc- 
curred. The notification may be received by an- 
other stream handier, or an application program. 

The application has the ability to tell data 
streaming routines (via Enable/Disable events) 
which events should be detected. These events 
typically are error detection notification, stream cue 
time events and data cue events etc. Also stream 
handlers may need to be informed of conditions of 
the data stream that may warrant certain actions to 
be preformed to keep the data stream running in a 
real time manner. To accomplish this, these events 
need to be routed to application or stream handler 
event routine in a timely (real time) manner so that 
the appropriate action can be preformed before a 
minimum amount of time has elapsed. To com- 
plicate this event reporting is the fact that the 
stream handlers can be running at ring 0 or ring 3. 

To report events from a ring 3 stream handler 
to a ring 3 stream handler or application is straight 
forward in that the registered event routine can be 
called in the context of the thread reporting the 
event. But when a ring 0 stream handler needs to 
report an event to a ring 3 stream handier or 
application, the problem arises in that the event 
can be detected by stream handler during interrupt 
time at ring 0 and there is no way to directly call 
the ring 3 event routine during this time. To solve 
this problem, a new time critical thread was created 
that has minimum task switching overhead and can 
be dispatched fast enough to allow an event to be 
reported in a real time environment Time critical 
threads are non-interruptable during execution. For 
each application process, this new time critical 
thread is created to monitor the events being re- 
ported. The sequence of events to report an event 
are as follows: 

1. The event is detected at interrupt time by the 
ring 0 stream handler. 

2. The event is put on a queue of events and 
the new time critical thread is unblocked to allow 
it to run. It will run in the context of the ring 3 
application. 

3. The time critical thread gets control in the 
context of the application, dequeues the event 
from the event queue and calls the applications 
event routine. 

This sequence of events is preformed in a 
minimum amount of time allowing the application to 
perform actions in real time thus making for a more 
robust and eye pleasing application. 



Claims 

1. A multimedia data processing system compris- 
ing a personal computer having a memory for 

5 storing at least one multimedia application pro- 

gram and a multitasking operating system, 
data streaming apparatus operable under said 
operating system for streaming data from a 
source device to a target device, said appara- 

70 tus comprising: 

first means adapted, in response to execu- 
tion of a stream create instruction in said ap- 
plication program, to create a source thread 
. and a target thread as multitask threads under . 

75 said operating system, to create a stream, and 

to allocate a plurality of buffers in said stream;. 

second means operative in response to a 
start instruction in said application program, to 
first run said source thread and fill a plurality of 

20 buffers with data from said source device, to 

then run said target thread and read data from 
said buffers into said target device, and to 
thereafter runs both threads by alternately fill- 
ing and reading said buffers until an end of 

25 data stream is reached. 

2. A multimedia data processing system in accor- 
dance with claim 1 wherein: 

said first means and said second means 
30 comprise a plurality of stream handlers, one of 

said stream handlers being a source handler 
operative to control data flow from said source 
device to said buffers, and another one of said 
stream handlers being a target stream handler 
35 operative to control data flow from said buffers 

to said target device. 

3. A multimedia data processing system in accor- 
dance with claim 2 comprising: 

40 a stream manager operative to interact 

with said stream handlers and control opera- 
tion thereof by sending stream helper com- 
mands (SHCs) to said handlers and by provid- 
ing stream manager helper (SMH) services to 

45 said handlers in response to receiving SMH 

calls from said handlers. 

4. A multimedia data processing system in accor- 
dance with claim 3 wherein: 

so said SHCs include a SHC. CREATE com- 

mand and a SHC. ST ART command; 

and each stream handler is operative in 
response to receiving a SHC. CREATE com- 
mand to create a thread and then to block 

55 such thread, said each stream handler being 

further operative in response to subsequently 
receiving a SHC. START command to unblock 
such thread and access said buffers in accor- 
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dance with whether such thread is a source 
thread or a target thread. 

5. A multimedia data processing system in accor- 
dance with claim 4 wherein: 

each stream handler during accessing said 
buffers sends a SMH. NOTIFY call to said sys- 
tem manager; 

and said system manager being operative 
to manage a buffer pool and, in response to 
receiving said SMH. NOTIFY call, to receive a 
full buffer and return an empty buffer when 
called by a source stream handler and to re- 
turn a full buffer and receive an empty buffer 
when called by a target stream handler. 

6. A multimedia data processing system in accor- 
dance with claim 4 wherein: 

said personal computer comprises a 
microprocessor having different priority levels 
of operation; 

and said stream handlers comprise dy- 
namic link library (DLL) stream handlers that 
execute at one priority level and physical de- 
vice driver (PDD) stream handlers that execute 
at a priority level higher than said one priority 
level. 

7. A multimedia data processing system in accor- 
dance with claim 6 wherein: 

said stream manager includes routines ex- 
ecuted at said priority levels in accordance 
with said priority level of the one of said 
stream handlers interacting therewith. 

8. A multimedia data processing system in accor- 
dance with claim 3 wherein: 

at least one of said stream handlers in- 
cludes event detection means operative in re- 
sponse to detecting an event to notify an event 
handling routine. 

9. A multimedia data processing system in accor- 
dance with claim 8 wherein: 

said one stream handler operates at prior- 
ity level 0 and comprises a time critical thread 
for notifying said event routine at a lower prior- 
ity level. 

10. A multimedia data processing system in accor- 
dance with any one of claims 3 to 9 wherein: 

said data from said source comprises in- . 
terleaved data contained in sequentially inter- 
leaved records; 

and said source handler fills each buffer 
with a plurality of said interleaved records. 



11. A multimedia data processing system in accor- 
dance with claim 10 wherein: 

said target handler is operative to transfer 
only certain ones of said records from each 
5 buffer. 

12. A multimedia data processing system in accor- 
dance with claim 10 wherein: 

said records contain two different types of 
70 data; 

and said system further comprises a sec- 
ond target handler operative to handle only 
one type of data, said first mentioned target 
handler being operative to handle the other 
75 type of data. 

13. A multimedia data processing system in accor- 
dance with claim 12 where records in each 
buffer are identified according to data types, 

20 and each of said target handlers deterrnines 

offsets into a buffer of each record containing 
data of the type being handled thereby. 

14. A multimedia data processing system in accor- 
25 dance with claim 5 wherein: 

said source thread blocks when an empty 
buffer is unavailable. 

15. A multimedia data processing system in accor- 
30 dance with claim 14 wherein: 

said target handler is operative to return an 
empty buffer to said system manager upon 
transferring data therein to said target device; 

and said stream manager unblocks said 
35 source thread in response to an empty buffer 

being returned by said target handler. 
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